Skip to content

feat(ci): parallelize browser regression lane - #795

Merged
devswha merged 3 commits into
mainfrom
cursor/parallel-browser-lane-1ffd
Oct 11, 2026
Merged

devswha merged 3 commits into
mainfrom
cursor/parallel-browser-lane-1ffd

Conversation

@devswha

@devswha devswha commented Oct 11, 2026

Copy link
Copy Markdown
Owner

Change

The browser regression lane runs ~20 Playwright scripts serially, taking ~476s and setting CI time (~8.3 min total for "Integration and browser"). Developer machines have many cores but the lane uses few.

This PR parallelizes the browser lane to cut both CI and local test time:

Local parallelization:

  • New ci-browser-parallel.ts runs scripts concurrently with a configurable worker pool
  • BROWSER_WORKERS env var controls worker count (default: min(cores/2, 6))
  • BROWSER_PARALLEL=1 in ci-browser.sh delegates to the parallel runner
  • Each worker gets isolated ports (via port: 0), temp dirs (mkdtempSync), and evidence subdirectories
  • Demo build is shared read-only across workers

CI parallelization:

  • Split "Integration and browser" into parallel jobs:
    • integration: herdr integration tests alone
    • browser-main: ui-regression.ts alone (the longest at ~198s)
    • browser-other: remaining 27 scripts in parallel with 4 workers
    • integration-browser: gate job that checks all above, preserving the required check name
  • Added docs-only change detection: skip heavy tests for pure documentation PRs

Shared state analysis:

  • Ports: Each script uses createServer({ port: 0 }) → safe for parallelization
  • Temp dirs: Each script creates its own mkdtempSync → safe
  • Herdr sessions: Scripts share herdr-web-ui-test but create their own workspaces → safe
  • Demo build: Read-only copy per script from shared HERDR_DEMO_BUILD → safe

Validation

Unit tests:

bun run test:unit
2549 pass, 8 skip, 0 fail

Local parallel test (8 demo scripts, 4 workers):

Run 1: 82.6s (vs ~165s serial estimate = 50% reduction)
Run 2: 82.4s (stable)

Example output:

Browser lane: 8 scripts, 4 workers
[worker-0] Starting scripts/composer-fit-demo-regression.ts
[worker-1] Starting scripts/droplet-demo-regression.ts
[worker-2] Starting scripts/chat-greeting-demo-regression.ts
[worker-3] Starting scripts/restored-draft-demo-regression.ts
[worker-2] PASS scripts/chat-greeting-demo-regression.ts (6.1s)
[worker-2] Starting scripts/held-rows-demo-regression.ts
...
Total: 8/8 passed in 82.6s with 4 workers

Expected CI timing improvement:

  • Current: ~8.3 min (integration + browser serial)
  • After: ~3.5 min (longest of: integration ~3.5min, ui-regression ~3.3min, other-browser ~2-3min)

CI workflow structure:

changes → windows-install (if code changed)
       → macos-session (if code changed)
       → fast (always)
       → integration (if code changed) ─┐
       → browser-main (if code changed) ├→ integration-browser (gate)
       → browser-other (if code changed)┘
Open in Web Open in Cursor 

Add parallel browser test execution to cut CI and local test time:

- Add ci-browser-parallel.ts: runs browser scripts concurrently with a
  configurable worker pool (BROWSER_WORKERS env var, default min(cores/2, 6))
- Update ci-browser.sh: BROWSER_PARALLEL=1 delegates to parallel runner
- Split CI 'Integration and browser' into parallel jobs:
  - integration: herdr integration tests (~3.5 min)
  - browser-main: ui-regression.ts alone (~3.3 min, the longest script)
  - browser-other: remaining 27 scripts in parallel (~2-3 min with 4 workers)
  - integration-browser: gate job for required check compatibility
- Add docs-only change detection: skip heavy tests for pure documentation PRs
- Update pages-gate.test.ts for new CI structure

Each worker gets isolated ports (via port: 0), temp dirs (mkdtempSync), and
evidence subdirectories. Demo build is shared read-only across workers.

Local testing (8 demo scripts, 4 workers):
- Serial: ~165s estimated
- Parallel: ~82s actual (50% reduction)

Scripts that need herdr share the test session but create their own workspaces,
allowing safe concurrent execution.

Co-authored-by: Hako <devswha@gmail.com>
@coderabbitai

coderabbitai Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 8 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: devswha/herdr-web-ui/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 622f2472-4fc1-4554-a0c7-5617846ab027

📥 Commits

Reviewing files that changed from the base of the PR and between e6febbe and 2890f7e.


📒 Files selected for processing (5)
  • .github/workflows/ci.yml
  • docs/development.md
  • scripts/ci-browser-parallel.ts
  • scripts/ci-browser.sh
  • scripts/pages-gate.test.ts

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

devswha and others added 2 commits October 11, 2026 02:06
…p full logs

The parallel runner and the browser-other job carried their own copies of
the serial script list, and both dropped sidebar-spacing-demo-regression.ts.
The runner now reads scripts/ci-browser.sh, browser-other excludes only
ui-regression.ts, and pages-gate.test.ts fails if the CI jobs do not run
the serial list exactly once. Unknown BROWSER_SCRIPTS/BROWSER_EXCLUDE
names are an error.

Each script's output now streams to <CHECK_DIR or .ci>/browser-logs/<script>.log
(with summary.json), a still-running line prints every minute, a failure
prints the last 400 lines plus the log path, and both browser jobs upload
the logs. docs/development.md describes the parallel jobs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@devswha
devswha marked this pull request as ready for review October 11, 2026 09:43
@devswha
devswha merged commit 4ed6c1b into main Oct 11, 2026
9 checks passed
@devswha
devswha deleted the cursor/parallel-browser-lane-1ffd branch October 11, 2026 09:43
devswha added a commit that referenced this pull request Oct 11, 2026
## Change

Release 0.4.7, from main at 80ca9bb (47 PRs since v0.4.6).

- `package.json` and `herdr-plugin.toml`: 0.4.7.
- CHANGELOG: `[Unreleased]` becomes `[0.4.7] - 2026-10-12`, with the
compare links.
- #513's entry moves out of the 0.4.0 section, where its branch's merge
left it, and gets its credit; it says the pace is a setting, off by
default (#834).
- A Maintenance section covers the PRs that had no entry: #743, #756,
#767, #795, #805, #810, #813, #821, #826, #831.
- The note under the Hermes entry that asked for a runtime bump at
release now says what a remote PC needs.
- `release-summaries.json`: 0.4.7 in en/ko/ja/zh/tr (6 new, 6 improved,
5 fixed). The Turkish lines are not checked by a native speaker.
- `REMOTE_BUNDLE_VERSION` 25 -> 26. Since v0.4.5's runtime the bridge
gained Hermes transcripts (#604), conversation pushes (#689) and
Grok/Antigravity activity (#816), so `remote-v26` is published before
the app release.

Not in this release: #606.

## Validation

- `bun scripts/release-notes.ts 0.4.7` renders the summaries and the
full changelog (235 lines).
- `bun run typecheck` and `bun run build`: pass.
- `TMPDIR=/tmp bun run test:unit`: one failure, `server/voice.test.ts`
("audio.webm" vs "clip.webm"), which fails the same way on unmodified
`main` on this Mac (Bun 1.4.0); CI runs Bun 1.4.2.
- Every PR merged since v0.4.6 was checked against the changelog
section: all 47 are linked.

## Release

After this merges and its CI on `main` passes: push `remote-v26`, wait
for its release, then run the Release workflow with `0.4.7`.
devswha added a commit that referenced this pull request Oct 13, 2026
…d runner (#873)

## Change

Review of CI after the recent merges (#795 browser lane parallelization,
#805 CI setup speedups), with two defects fixed and an opt-in
self-hosted runner for Linux CI/CD.

### Fixes
- **Release/Pages validation could skip heavy tests.** Under
`workflow_call` (release.yml, pages.yml) and `workflow_dispatch`, Detect
changes diffed `HEAD~1`; a docs-only last commit set `code=false` and
the release was validated without integration and browser tests. Only a
`pull_request` may now skip them.
- **Docs-only regex.** `^(docs/|\.md$|...)` anchored `\.md$` to the path
start, so no `*.md` outside docs/ ever counted as docs. Now
`^docs/|\.(md|txt)$|^LICENSE$|^\.github/ISSUE_TEMPLATE/`.
- `${{ }}` values in `run:` scripts go through `env:`.
- The Bun/Node/herdr setup duplicated across 7 jobs is one composite
action (`.github/actions/setup-ci`); the Bun version is set once
(pages.yml and release.yml pinned it separately).

### Self-hosted runner (opt-in, default unchanged)
- Repository variable `CI_SELF_HOSTED_LABELS` (JSON array containing
`"self-hosted"`) routes Fast checks, Integration tests, both Browser
regression jobs, Pages build/deploy and Release preflight/publish. Unset
= `ubuntu-24.04` as today. ci.yml also takes a `self_hosted_labels`
dispatch input for one manual run.
- Fork PRs always get `ubuntu-24.04` (`scripts/ci-runner-route.sh`,
tested in `scripts/ci-runner-route.test.ts`). Since a fork PR can edit
`runs-on` itself, REVIEW.md now says not to approve such runs.
- `scripts/self-hosted-runner/`: an ephemeral Docker runner (one job per
container, `--cap-drop ALL`, `no-new-privileges`, no sudo, no host
mounts, registration token on stdin) with Chromium libraries and ffmpeg
baked in. The browser lane skips its sudo `install-deps` when
`RUNNER_ENVIRONMENT=self-hosted`.
- Note: GitHub-hosted minutes are free for this public repository, so
the gain is speed (a 32-core host vs 4-core runners), not cost.

## Validation
- `bun run check fast`: workflow syntax, generated types, typecheck,
build, unit (2952 pass, 0 fail).
- Route script cases (fork PR, variable, dispatch override, malformed
labels) in `scripts/ci-runner-route.test.ts`.
- Runner image: uid 1001, no sudo, Chromium launches without root under
`--cap-drop ALL`; token absent from `/proc/1/environ`.
- A workflow_dispatch run of this branch on the self-hosted runner: see
the comment below.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants